Description

Add an ORT_ENFORCE check in the QMoECPU constructor to require swiglu_fusion == 1 when using SwiGLU activation, preventing an out-of-bounds read.

When swiglu_fusion=0 (the default), fc1_out_features is computed as inter_size instead of 2*inter_size. However, ApplySwiGLUActivation reads 2*inter_size values from the FC1 output buffer (via input_data[2*i] for i in [0, inter_size)), causing an out-of-bounds read that produces NaN on Windows x86.

This matches the existing validation already present in the MoE CPU operator (moe_cpu.cc:26-27).

Motivation and Context

• The NaN was caused by a missing swiglu_fusion=1 attribute. With the default swiglu_fusion=0, the SwiGLU activation reads past the allocated FC1 output buffer — an out-of-bounds read.
• The MoE CPU operator already enforces swiglu_fusion == 1 for SwiGLU; this change adds the same guard to QMoECPU for consistency and safety.
• Non-interleaved SwiGLU format (swiglu_fusion=2) is not implemented (throws ORT_NOT_IMPLEMENTED), and swiglu_fusion=0 is invalid for SwiGLU, so only swiglu_fusion=1 is valid.